Apparatus and method for estimating bio-information

ABSTRACT

An apparatus for non-invasively estimating bio-information includes an image sensor configured to acquire a first contact image of an object, based on the object being in contact with the image sensor, an actuator, and a processor configured to determine a contact position and a direction of the object, based on the acquired first contact image, and control the actuator to adjust a position of the image sensor, based on the determined contact position and the determined direction of the object so that a field of view (FOV) of the image sensor moves to a predefined measurement area on the object.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority under 35 U.S.C. § 119 to Korean PatentApplication No. 10-2020-0135936, filed on Oct. 20, 2020, in the KoreanIntellectual Property Office, the disclosure of which is incorporated byreference herein in its entirety.

BACKGROUND 1. Field

Apparatuses and methods consistent with example embodiments relate tonon-invasively estimating bio-information.

2. Description of Related Art

Methods of non-invasively measuring blood pressure without damaging ahuman body include a method of measuring blood pressure by measuring acuff-based pressure and a method of estimating blood pressure bymeasuring a pulse wave without the use of a cuff.

A Korotkoff-sound method is one of cuff-based blood pressure measurementmethods, in which a pressure in a cuff wound around an upper arm isincreased and blood pressure is measured by listening to the soundgenerated in the blood vessel through a stethoscope while decreasing thepressure. Another cuff-based blood pressure measurement method is anoscillometric method using an automated machine, in which a cuff iswound around an upper arm, a pressure in the cuff is increased, apressure in the cuff is continuously measured while the cuff pressure isgradually decreased, and blood pressure is measured based on a point atwhich a change in a pressure signal is large.

Cuffless blood pressure measurement methods may include a method ofmeasuring blood pressure by calculating a pulse transit time (PTT) and apulse wave analysis (PWA) method of estimating blood pressure byanalyzing a shape of a pulse wave.

SUMMARY

In accordance with an aspect of an example embodiment, there is providedan apparatus for estimating bio-information including an image sensorconfigured to acquire a first contact image of an object, based on theobject being in contact with the image sensor, an actuator, and aprocessor configured to determine a contact position and a direction ofthe object, based on the acquired first contact image, and control theactuator to adjust a position of the image sensor, based on thedetermined contact position and the determined direction of the objectso that a field of view (FOV) of the image sensor moves to a predefinedmeasurement area on the object.

The processor may be further configured to extract a characteristicpoint of the object, from the acquired first contact image, anddetermine the contact position and the direction, based on the extractedcharacteristic point.

The characteristic point of the object may include a fingerprint centerpoint of a finger.

The processor may be further configured to determine whether theextracted characteristic point exists in the first contact image, andbased on the extracted characteristic point being determined to notexist in the first contact image, guide a user to bring the object intocontact with the image sensor.

The processor may be further configured to determine whether theextracted characteristic point exists in the first contact image, andbased on the extracted characteristic point being determined to notexist in the first contact image, repeat a predetermined number of timesof controlling the actuator to adjust the position of the image sensorto an arbitrary position and then extracting the characteristic point,from the acquired first contact image acquired after the position of theimage sensor is adjusted.

The processor may be further configured to determine whether theextracted characteristic point exists in the first contact image, andbased on the extracted characteristic point being determined to notexist in the first contact image, estimate a position of thecharacteristic point by comparing a reference contact image and theacquired first contact image.

The processor may be further configured to determine a displacement ofthe image sensor, based on the determined contact position and thedetermined direction, and control the actuator to adjust the position ofthe image sensor, based on the determined displacement.

The image sensor may be further configured to acquire a second contactimage of the object, based on the position of the image sensor beingadjusted, and the processor may be further configured to extract a pulsewave signal, based on a pixel intensity of the acquired second contactimage, and estimate the bio-information, based on the extracted pulsewave signal.

The processor may be further configured to generate an oscillogram,based on the extracted pulse wave signal and a contact pressure of theobject, and estimate the bio-information, based on the generatedoscillogram.

The apparatus may further include a force/pressure sensor configured tomeasure a contact force or the contact pressure that is applied betweenthe object and the image sensor, based on the object in contact with theimage sensor changing a force.

The processor may be further configured to acquire the contact pressure,based on the pixel intensity of the acquired second contact image andusing a predefined contact pressure conversion equation.

The bio-information may include any one or any combination of a bloodpressure, a vascular age, an arterial stiffness, an aortic pressurewaveform, a blood vessel elasticity, a stress index, and a degree offatigue.

In accordance with an aspect of an example embodiment, there is provideda method of estimating bio-information, the method including acquiring,by an image sensor, a first contact image of an object, based on theobject being in contact with the image sensor, determining, by aprocessor, a contact position and a direction of the object, based onthe acquired first contact image, and controlling, by the processor, anactuator to adjust a position of the image sensor, based on thedetermined contact position and the determined direction of the objectso that a field of view (FOV) of the image sensor moves to a predefinedmeasurement area on the object.

The determining of the contact position and the direction may includeextracting a characteristic point of the object, from the acquired firstcontact image, and determining the contact position and the direction,based on the extracted characteristic point.

The method may further include determining whether the extractedcharacteristic point exists in the first contact image, and based on theextracted characteristic point being determined to not exist in thefirst contact image, guiding a user to bring the object into contactwith the image sensor.

The method may further include determining whether the extractedcharacteristic point exists in the first contact image, and based on theextracted characteristic point being determined to not exist in thefirst contact image, repeating a predetermined number of times ofcontrolling the actuator to adjust the position of the image sensor toan arbitrary position and then extracting the characteristic point, fromthe acquired first contact image acquired after the position of theimage sensor is adjusted.

The method may further include determining whether the extractedcharacteristic point exists in the first contact image, and based on theextracted characteristic point being determined to not exist in thefirst contact image, estimating a position of the characteristic pointby comparing a reference contact image and the acquired first contactimage.

The controlling of the actuator may include determining a displacementof the image sensor, based on the determined contact position and thedetermined direction, and controlling the actuator to adjust theposition of the image sensor, based on the determined displacement.

The method may further include acquiring a second contact image of theobject, based on the position of the image sensor being adjusted,extracting a pulse wave signal, based on a pixel intensity of theacquired second contact image, and estimating bio-information, based onthe extracted pulse wave signal.

The estimating of the bio-information may include generating anoscillogram, based on the extracted pulse wave signal and a contactpressure of the object, and estimating the bio-information, based on thegenerated oscillogram.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of embodiments ofthe disclosure will be more apparent from the following descriptiontaken in conjunction with the accompanying drawings, in which:

FIG. 1 is a block diagram illustrating an apparatus for estimatingbio-information according to an example embodiment.

FIG. 2 is a block diagram illustrating an apparatus for estimatingbio-information according to another example embodiment.

FIG. 3 is a block diagram illustrating an apparatus for estimatingbio-information according to still another example embodiment.

FIGS. 4A, 4B, 4C, 4D, 4E, 4F, 4G and 4H are diagrams for describing astructure of a sensor and image sensor position adjustment.

FIGS. 5A and 5B are graphs for describing oscillometric-based bloodpressure estimation.

FIG. 6 is a flowchart illustrating a method of estimatingbio-information according to an example embodiment.

FIG. 7 is a flowchart illustrating a method of estimatingbio-information according to another example embodiment.

FIG. 8 is a flowchart illustrating a method of estimatingbio-information according to still another example embodiment.

DETAILED DESCRIPTION

Details of example embodiments are provided in the following detaileddescription with reference to the accompanying drawings. The disclosuremay be understood more readily by reference to the following detaileddescription of the example embodiments and the accompanying drawings.The disclosure may, however, be embodied in many different forms and maynot be construed as being limited to the embodiments set forth herein.Rather, these embodiments are provided so that the disclosure will bethorough and complete and will fully convey the concept of the inventionto those skilled in the art, and the disclosure will only be defined bythe appended claims. Like reference numerals refer to like elementsthroughout the specification.

It will be understood that, although the terms first, second, etc. maybe used herein to describe various elements, these elements may not belimited by these terms. These terms are only used to distinguish oneelement from another. Also, the singular forms are intended to includethe plural forms as well, unless the context clearly indicatesotherwise. In the specification, unless explicitly described to thecontrary, the word “comprise” and variations such as “comprises” or“comprising,” will be understood to imply the inclusion of statedelements but not the exclusion of any other elements. Terms such as“unit” and “module” denote units that process at least one function oroperation, and they may be implemented by using hardware, software, or acombination of hardware and software.

Hereinafter, embodiments of an apparatus and method for estimatingbio-information will be described in detail with reference to thedrawings.

Embodiments of an apparatus for estimating bio-information describedherein may be included in various information processing devices, suchas a portable wearable device, a smart device, and the like. Forexample, various information processing devices may include varioustypes of wearable devices, such as a smartwatch worn on a wrist, a smartband type wearable device, a headphone-type wearable device, and a hairband type wearable device, and mobile devices, such as a smartphone, atablet personal computer (PC), etc. However, the information processingdevice is not limited to the above examples.

FIG. 1 is a block diagram illustrating an apparatus 100 for estimatingbio-information according to an example embodiment.

Referring to FIG. 1, the bio-information estimation apparatus 100includes an image sensor 110, an actuator 120, and a processor 130.

The image sensor 110 may optically acquire a contact image of an object.The image sensor 110 may include an optical-based image sensor, such asa CMOS Image Sensor (CIS), or a fingerprint sensor. However, aspects ofthe disclosure are not limited thereto, such that an array of, forexample, photodiodes or photo transistors, may be formed, rather thanthe image sensor 110, a light signal detected from each pixel may beconverted into an electrical signal, such as an electric charge or avoltage signal, and the electrical signal of each pixel may be output aspixel data.

The apparatus 100 for estimating bio-information may include a lightsource to emit light to the object when the object is in contact withthe image sensor 110. The light source may include one or morelight-emitting didoes (LEDs), laser diodes (LDs), phosphors, and thelike, but is not limited thereto. Alternatively, instead of including aseparate light source, light input from the outside may be used as alight source.

The actuator 120 may adjust a position of the image sensor 110 under thecontrol of the processor 130. There is no particular limitation on thedriving method of the actuator 120, such as a motor-based method, anencoder-based method, a piezo-based method, or the like. A maximum fieldof measurement (hereinafter referred to as “FOM”) in which the actuator120 can move the image sensor 110 to perform measurement may be presetand be stored in a memory.

The processor 130 may drive the actuator 120 on the basis of the contactimage of the object acquired by the image sensor 110. The processor 130may determine a current contact position and a current direction of theobject by analyzing the contact image of the object. Also, when thecurrent contact position and direction of the object is determined, theprocessor 130 may control the actuator 120 to move to a field of view(hereinafter referred to as “FOV”) currently indicated by the imagesensor 110 to a predefined desired measurement area on the object, onthe basis of the determined current contact position and direction.

For example, the processor 130 may extract a characteristic point of theobject from the contact image of the object and determine the currentposition and direction of the object based on the extractedcharacteristic point. For example, the processor 130 may determine thecurrent position and direction by comparing the characteristic point ofthe object and a center point of a current FOV of the image sensor 110.In this case, when the object is a finger, the characteristic point mayinclude a center point of a fingerprint of the finger. However, aspectsof the disclosure are not limited thereto, and the object may be, forexample, a predefined position of a blood vessel on the object. Even forthe same object, the position of a blood vessel may be slightlydifferent according to characteristics of each user, and thus can beacquired in advance for each user through a preprocessing process.

The processor 130 may calculate a displacement by which to move theimage sensor 110 on the basis of the determined current contact positionand direction of the object and control the image sensor 110 to acquirea contact image from a new position, i.e., the desired measurement areaby driving the actuator according to the calculated displacement.

If the characteristic point does not exist in the contact image of theobject, the processor 130 may guide the user to bring the object againinto contact with the image sensor 110.

Also, when no characteristic point exists in the contact image of theobject, the processor 130 may drive the actuator 120 to move the imagesensor 110 to an arbitrary position for a predefined number of timesinstead of guiding the user to immediately re-contact, and may repeatthe operation of extracting a characteristic point from a positionadjusted by the image sensor 110 on the basis of the acquired contactimage. When a characteristic point is not extracted even after apredetermined number of repetitions as described above, the processor130 may guide the user to bring the object again into contact with theimage sensor 110. In this case, the processor 130 may determine thearbitrary position to which the image sensor is to be moved by takinginto account a relative position of the current FOV in the range of theFOM of the actuator.

In addition, when no characteristic point exists in the contact image ofthe object, the processor 130 may estimate a position of acharacteristic point by comparing the contact image and a referencecontact image, and move the FOV of the image sensor 110 to the estimatedposition of a characteristic point. In this case, the reference contactimage may be a contact image acquired in a state in which the FOV of theimage sensor 110 matches a measurement position of an object predefinedfor each use during the process of calibration. For example, afingerprint image measured in advance from a user's finger may be set asa reference fingerprint image, and the position of a characteristicpoint may be estimated by comparing fingerprint patterns of a currentlymeasured fingerprint image of the finger and the reference fingerprintimage. At this time, the reference fingerprint image may be stored in astorage.

When the contact image is acquired in the desired measurement area ofthe object by driving the actuator, the processor 130 may estimatebio-information based on pixel data of the contact image. In this case,the bio-information may include blood pressure, vascular age, arterialstiffness, aortic pressure waveform, blood vessel elasticity, stressindex, and a degree of fatigue, but is not limited thereto. Hereinafter,for convenience of description, blood pressure will be taken as anexample.

When it is determined that the current FOV of the image sensor 110matches the desired measurement area of the object, the processor 130may estimate bio-information on the basis of the acquired currentcontact image without driving the actuator 120. For example, when adistance between the center point of the current FOV of the image sensor110 and a fingerprint characteristic point is less than or equal to apredetermined threshold, it may be determined that the current FOVmatches the desired measurement area of the object.

The processor 130 may extract a pulse wave signal on the basis of pixeldata of the image sensor 110, i.e., the intensity of light received byeach pixel while the object in contact with the image sensor 110 changesa contact pressure for a predetermined period of time, and may estimateblood pressure on the basis of the extracted pulse wave signal. In thiscase, the image sensor 110 may define an electrical signal value of eachpixel as the intensity of each pixel. The intensity of each pixel mayvary depending on the time for which the object is in contact with theimage sensor 110, the area in which the object is in contact with theimage sensor 110, or the like. For example, when the user brings his/herfinger into contact with the image sensor 110 and gradually increasesthe pressing force for a predetermined period of time, the contact timeand the contact area gradually increase and the intensity of each pixelincreases. Hence, it can be seen that there is a correlation between theintensity of each pixel and the amplitude of the pulse wave signal.

For example, the processor 130 may convert the intensity of each pixelat each point in time into an amplitude value of a pulse wave signal atthat point in time using a predefined amplitude conversion equation. Theamplitude conversion equation may be a function equation that outputsany one of an average, a median, a minimum value, and a maximum value ofpixel intensities as an amplitude value. However, aspects of thedisclosure are not limited thereto, and the amplitude conversionequation may be defined as various liner or non-linear functionequations.

The processor 130 may extract a feature related to blood pressurethrough, for example, an analysis of a waveform of a pulse wave signal,and estimate blood pressure using a predefined blood pressure estimationmodel. In this case, the feature may include time and/or amplitude of amaximum point of an amplitude of a pulse wave signal, a propagation waveconstituting a pulse wave signal, time and/or amplitude of a pulsewaveform related to a reflection wave, area information of a waveform ofa pulse wave signal, and the like.

In another example, an oscillogram may be generated as described below,and blood pressure may be estimated by using the generated oscillogram.In this case, the processor 130 may convert the intensity of each pixelinto a contact pressure by using a predefined contact pressureconversion equation. That is, as described above, as the contactpressure increases, the contact area increases and thus the intensity ofeach pixel increases. Accordingly, it can be seen that there is acorrelation between the contact pressure and the intensity of eachpixel. As described above, the contact pressure conversion equation thatdefines a correlation between the contact pressure and the intensity ofeach pixel may be predefined.

FIG. 2 is a block diagram illustrating an apparatus 200 for estimatingbio-information according to another example embodiment.

Referring to FIG. 2, the apparatus 200 for estimating bio-informationmay include an image sensor 110, an actuator 120, a processor 130, and aforce/pressure sensor 210. The image sensor 110, the actuator 120, andthe processor 130 are described above, and hence detailed descriptionsthereof will not be reiterated.

The force/pressure sensor 210 may be disposed on a lower portion of theimage sensor 110 and measure a contact force or a contact pressureapplied by an object to the image sensor 110. The force/pressure sensor210 may include a single force sensor, such as a strain gauge, a forcesensor array, a combination of a force sensor and an area sensor, or apressure sensor. For example, the force sensor 210 may be a voltageresistance type force sensor, an ultrasonic type force sensor, a loadcell sensor, a capacitive force sensor, a pyroelectric force sensor, astrain gauge type force sensor, an electrochemical force sensor, anoptical force sensor, or a magnetic type force sensor.

When the contact force or contact pressure between the object and theimage sensor 110 is acquired through the force/pressure sensor 210, theprocessor 130 may estimate blood pressure through oscillometry on thebasis of a pulse wave signal extracted based on the contact force orcontact pressure and the pixel intensity of the image sensor 110. Inthis case, when the contact force is measured, the contact force may beconverted into a contact pressure based on the area of the image sensor110.

FIG. 3 is a block diagram illustrating an apparatus 300 for estimatingbio-information according to another example embodiment.

Referring to FIG. 3, the apparatus 300 for estimating bio-informationmay include an image sensor 110, an actuator 120, a processor 130, anoutput interface 310, a storage 320, and a communication interface 330.The apparatus 300 may further include a force/pressure sensor 210. Theimage sensor 110, the actuator 120, and the processor 130 are describedabove, and hence detailed descriptions thereof will not be reiterated.

The output interface 310 may provide a user with data generated duringthe process of estimating blood pressure by the processor 130. Forexample, the output interface 310 may display a contact image of anobject or a blood pressure estimation result on a display. In this case,when an estimated blood pressure value is out of a normal range, warninginformation may be provided to the user by adjusting color or thicknessof a line so that the user can easily recognize it or by being displayedtogether with the normal range. In addition, the output interface 310may provide information related to the estimated blood pressure value tothe user through a voice output interface, a haptic interface, or thelike, together with or independently of a visual display, in anon-visual method, such as voice, vibration, tactile sensation, etc.

The storage 320 may store data related to the blood pressure estimation.For example, the storage 320 may store the contact image acquiredthrough the image sensor 110, pixel data, various data generated in theblood pressure estimation process by the processor 130, for example, anextracted characteristic point, an estimated blood pressure value, orthe like. Also, the storage 320 may store reference information, such asa blood pressure estimation model related to the blood pressureestimation, a contact pressure conversion equation, an amplitudeconversion equation, or the like.

The storage 320 include at least one type of storage medium, such as aflash memory type, a hard disk type, a multimedia card micro type, acard type memory (e.g., a secure digital (SD) or eXtreme digital (XD)memory), a random access memory (RAM), a static random access memory(SRAM), a read-only memory (ROM), an electrically erasable programmableread-only memory (EEPROM), a programmable read-only memory (PROM), amagnetic memory, a magnetic disk, an optical disk, and the like, but isnot limited thereto.

The communication interface 330 may communicate with an external deviceto transmit and receive various types of data related to the bloodpressure estimation. The external device may include an informationprocessing device, such as a smartphone, a tablet PC, a desktop PC, alaptop PC, and the like. For example, a blood pressure estimation resultmay be transmitted to the external device, such as a user's smartphone,so that the user can manage and monitor a component analysis resultthrough a device which has a relatively high performance. In addition,information, such as a blood pressure estimation model, reference bloodpressure, and the like, may be received from the external device and bestored in the storage 320.

The communication interface 330 may communicate with the external deviceby using various wired or wireless communication techniques includingBluetooth communication, Bluetooth low energy (BLE) communication, nearfield communication (NFC), wireless local access network (WLAN)communication, ZigBee communication, infrared data association (IrDA)communication, Wi-Fi Direct (WFD) communication, ultra-wideband (UWB)communication, Ant+ communication, Wi-Fi communication, radio frequencyidentification (RFID) communication, 3G communication, 4G communication,and/or 5G communication. However, the communication techniques are notlimited thereto.

FIGS. 4A, 4B, 4C, 4D, 4E, 4F, 4G and 4H are diagrams for describing astructure of a sensor 41 and an example of image sensor positionadjustment.

Example embodiments of a structure of the sensor and image sensorposition adjustment will be described with reference to FIGS. 1 and 4Ato 4H.

FIG. 4A illustrates a smart device 40 as one example of devices equippedwith the apparatus 100 for estimating bio-information. As illustrated,the image sensor 110 of the apparatus 100 for estimating bio-informationmay be mounted in the sensor 41 on the rear surface of the smart device40. However, aspects of the disclosure are not limited thereto, and afingerprint sensor on the front surface for performing fingerprintauthentication or a front-surface image sensor on may perform thefunction of the image sensor 110.

In the sensor 41, a filter array including a color filter for passing orblocking light of a wavelength range t may be arranged above each pixelof the image sensor 110. In addition, a lens for gathering lightscattered, reflected, or transmitted from the object and directing thelight to the image sensor 110 may be disposed in the sensor 41. Also, amicro lens for increasing the light-gathering ability may be disposedabove each pixel of the image sensor 110. In this case, light emittedfrom the surrounding environment of the smart device 40 may be used asan external light source. Alternatively, a separate internal lightsource may be mounted around or inside the sensor 41 on the rear surfaceof the smart device 40.

FIG. 4B illustrates a structure in which the image sensor 110 isdisposed in the sensor 41 and the actuator 120 for adjusting a positionof the image sensor 110 is disposed around the image sensor 110. Asillustrated, the FOV of the image sensor 110 may be configured to bemovable within the FOM according to the driving of the actuator 120.

FIGS. 4C to 4E schematically illustrate structures for describing anarrangement relationship between the components of the sensor 41.However, aspects of the disclosure are not limited to the examplesillustrated herein, and the structures may be variously changedaccording to the size, shape, or the like of a form factor. Referring toFIG. 4C, a lens 113 for gathering light from the object is disposed atan upper side, and an optical part 111 in which the image sensor 110 andone or more light sources 112 are disposed may be positioned at a lowerside thereof. In this case, the actuator 120 may be disposed around theoptical part 111 and adjust a position of the optical part 111, therebyadjusting a position of the image sensor 110. Also, the force sensor 210may be disposed at a lower side of the optical part 111. The forcesensor 210 may include, for example, a load cell 211. Referring to FIG.4D, the lens 113 for gathering light from the object may be disposed atan upper side, and the optical part 111 may be disposed at a lower sidethereof. In this case, the actuator 120 may be disposed at a lower sideof the optical part 111 and adjust a position of the optical part 111.The force sensor 210 may be disposed at a lower side of the actuator120. Referring to FIG. 4E, the lens 113, the optical part 111, and theforce sensor 210 may be vertically disposed. In this case, the actuator120 may be disposed on a side surface of the optical part 111 and theforce sensor 210 and simultaneously move the positions of the opticalpart 111 and the force sensor 210.

Examples of adjusting the position of the image sensor will be describedwith reference to FIGS. 4F to 4H.

Referring to FIG. 4F, portion (1) illustrates that a finger isaccurately in contact with the FOM of the image sensor 110, but the FOVof the image sensor 110 is skewed to the lower left side of the FOM sothat a fingerprint image is acquired from the left lower side of afinger. The processor 130 may extract a characteristic point, forexample, a fingerprint center point 42, from the fingerprint image inthe current FOV. In addition, the processor 130 may compare a centerpoint 43 of the FOV and the extracted fingerprint center point 42 anddetermine that a position of the object in currently contact with theimage sensor 110 is in the lower left of the finger. When the contactposition and direction are determined in this way, the processor 130 maycalculate a displacement 44 by which to move the FOV of the image sensor110, and control the actuator 120 by using the calculated displacement.In this case, the displacement may include information on a direction inwhich to move and a distance to move, and the displacement may becalculated so that the center of the FOV of the image sensor 110 matchesthe fingerprint center point.

Portion (2) of FIG. 4F illustrates that the FOV of the image sensor 110is moved and matches a desired measurement area MA of the object as theprocessor 130 drives the actuator 120 as described above. In addition,portion (3) of FIG. 4F illustrates that the finger is accurately incontact with the FOM of the image sensor 110, but the FOV of the imagesensor 110 is skewed to the upper right side of the FOM so that afingerprint image is acquired from the upper right side of the finger.Likewise, the processor 130 may compare the extracted fingerprint centerpoint 42 and the center point 43 of the FOV to calculate a displacement44, and drive the actuator 120 according to the calculated displacement44, thereby allowing the FOV to match the desired measurement area MA asshown in portion (2).

FIG. 4G illustrates that the fingerprint center point does not existwithin the FOV of the image sensor 110 in a case in which a contactposition of the finger is not appropriate and is thus skewed toward theside of the FOM of the image sensor 110. The processor 130 may extract afingerprint center point 42 from a fingerprint image within the currentFOV. Because the fingerprint center point 42 exists within the currentFOV, the processor 130 may compare the extracted fingerprint centerpoint 42 and the center point 43 of the FOV and determine that thecurrent measurement area is in the right side of the finger. When thecontact position and direction are determined in this way, the processor130 may calculate a displacement 44 and control the actuator 120 byusing the calculated displacement 44, thereby allowing the FOV to matchthe desired measurement area MA of the finger as shown in portion (2).

According to the disclosed embodiments, the FOV of the image sensor 110is moved via the actuator 120 regardless of whether the contact positionof the finger accurately contacts the FOM of the image sensor 110, and asignal is measured at the desired measurement area of the object,thereby improving the accuracy of bio-information estimation.

FIG. 4H illustrates that the fingerprint center point does not exist inthe FOV of the image sensor 110 due to the improper contact position ofthe finger and thus skewed toward the side of the FOM of the imagesensor 110. The processor 130 may guide the user to bring the fingeragain into contact with the image sensor 110 when the fingerprint centerpoint 42 does not exist in the FOV. Alternatively, the processor 130 mayrepeat the process of extracting the fingerprint center point 42 bymoving the FOV to an arbitrary position a predefined number of times.

For example, the processor 130 may analyze a fingerprint pattern fromthe contact image within the FOV and determine the arbitrary position towhich to move the FOV. For example, because a fingerprint patternlocated only on the left side of the center point 43 of the FOV asillustrated, it may be assumed that the fingerprint center point ispositioned further left, and the FOV may be moved by a predetermineddisplacement in the left direction. Alternatively, a partial fingerprintimage 45 partially overlapping the FOV may be compared with thereference fingerprint image to estimate a portion of the reference imagewhich corresponds to the partial fingerprint image, a position of thefingerprint center point 42 may be estimated according to the estimationresult and a direction of movement may be determined.

FIGS. 5A and 5B are graphs for describing an example ofoscillometric-based blood pressure estimation. FIG. 5A shows a change inamplitude of a pulse wave signal while the object in contact with theimage sensor 110 gradually increases pressure. FIG. 5B shows anoscillogram OW showing a relationship between the change in contactpressure and the amplitude of the pulse wave signal.

The processor 130 may extract a peak-to-peak point by, for example,subtracting an amplitude value in3 of a negative (−) point of a pulsewave signal waveform envelope in1 from an amplitude value in2 of apositive (+) point at each measurement time point. In addition, anoscillogram OW may be obtained by plotting a peak-to-peak amplitudebased on a contact pressure value at the corresponding time point andperforming, for example, polynomial curve fitting.

The processor 130 may estimate blood pressure using the oscillogram OWgenerated as described above. For example, mean arterial pressure (MAP)may be estimated based on a contact pressure MP at a pulse wave maximumpoint MA in the oscillogram. For example, the contact pressure MP at thepulse wave maximum point MA may be determined as MAP. Alternatively, theMAP may be estimated by applying the contact pressure to a predefinedMAP estimation equation. In this case, the MAP estimation equation maybe defined as various linear or non-linear combination functions, suchas addition, subtraction, division, multiplication, logarithmic value,regression equation, and the like, with no specific limitation.

In addition, the processor 130 may estimate diastolic blood pressure andsystolic blood pressure based, respectively, on contact pressure valuesDP and SP at points on the left and right of the pulse wave maximumpoint MA, at each of which an amplitude has a value equal to apredetermined ratio (e.g., 0.5 to 0.7) to the amplitude value at thepulse wave maximum point MA. Likewise, the contact pressures DP and SPmay be determined as diastolic blood pressure and systolic bloodpressure, respectively, and the diastolic blood pressure and thesystolic blood pressure may be estimated using a predefined diastolicblood pressure estimation equation and a predefined systolic bloodpressure estimation equation.

FIG. 6 is a flowchart illustrating a method of estimatingbio-information according to an example embodiment.

The method of FIG. 6 may correspond to one embodiment of methodperformed by the apparatuses 100, 200, 300 for estimatingbio-information according to the embodiments of FIGS. 1 to 3. The methodis described in detail above, and hence will be briefly describedhereinafter.

In operation 610, when an object is in contact with the image sensor,the image sensor may acquire a contact image of the object.

In operation 620, the processor may determine a contact position anddirection of the object on the basis of the contact image of the objectacquired in operation 610. For example, the processor may extract acharacteristic point of the object from the contact image and determinethe current position and direction of the object based on the extractedcharacteristic point. For example, the processor may determine thecurrent position and direction by comparing the characteristic point ofthe object and a center point of a current FOV of the image sensor.

In operation 630, based on the determined contact position anddirection, it may be determined whether a position of the image sensoris to be adjusted. For example, if a distance between the center pointof the current FOV of the image sensor and a fingerprint characteristicpoint is less than a predetermined threshold, it may be determined thatthe current FOV matches the desired measurement area of the object, andotherwise, it may be determined that the position of the image sensor isto be adjusted.

When it is determined in operation 630 that the position of the imagesensor is to be adjusted, in operation 640, an actuator is driven toadjust the position of the image sensor, and the flowchart returns tooperation 610 to acquire a contact image of the object at the adjustedposition. In this case, a displacement of the image sensor may becalculated based on the characteristic point of the object and theactuator may be driven based on the calculated displacement.

When it is determined in operation 630 that the position of the imagesensor is not to be adjusted, in operation 650, a pulse wave signal maybe extracted based on the contact image acquired in operation 610. Forexample, an amplitude value of a pulse wave signal may be acquired basedon the intensity of light received by each pixel of the image sensor fora predetermined period of time.

In operation 660, bio-information may be estimated based on theextracted pulse wave signal. For example, blood pressure may beestimated through oscillometry on the basis of the pulse wave signal anda contact pressure. In this case, the contact pressure may be measuredby a force/pressure sensor, or may be acquired through conversion basedon the intensity of light received by each pixel of the image sensor. Anestimated bio-information value acquired as described above may beoutput to a user.

FIG. 7 is a flowchart illustrating a method of estimatingbio-information according to an example embodiment.

The method of FIG. 7 may correspond to another embodiment of the methodperformed by the apparatuses 100, 200, and 300 for estimatingbio-information according to the embodiments of FIGS. 1 to 3. The methodis described in detail above, and hence will be briefly describedhereinafter.

In operation 711, when an object is in contact with the image sensor,the image sensor may acquire a contact image of the object.

In operation 712, the processor may extract a characteristic point fromthe acquired contact image of the object. In this case, thecharacteristic point is a reference point within a desired measurementarea of the object and may include, for example, a center point of afingerprint or a position of a predetermined blood vessel within themeasurement area.

When the characteristic point is not extracted in operation 712, thatis, when the characteristic point does not exist in the FOV of the imagesensor (operation 713—No), in operation 714, a user may be guided forrecontact. When the characteristic point is extracted in operation 712(operation 713—Yes), in operation 715, the contact position anddirection of the object may be determined based on the extractedcharacteristic point. In this case, the contact position and directionmay be determined based on the extracted characteristic point and thecenter point of the FOV of the image sensor.

In operation 716, based on the determined contact position anddirection, it is determined whether the position of the image sensor isto be adjusted.

When the position of the image sensor is to be adjusted, in operation717, the position of the image sensor may be adjusted by driving theactuator. In this case, a displacement of the image sensor may becalculated based on the characteristic point of the object and theactuator may be driven based on the calculated displacement.

When it is determined in operation 716 that the position of the imagesensor is not to be adjusted, in operation 718, a pulse wave signal maybe extracted based on the contact image, and in operation 719,bio-information may be estimated based on the extracted pulse wavesignal.

FIG. 8 is a flowchart illustrating a method of estimatingbio-information according to an example embodiment.

The method of FIG. 8 may correspond to another embodiment of the methodperformed by the apparatuses 100, 200, and 300 for estimatingbio-information according to the embodiments of FIGS. 1 to 3. The methodis described in detail above, and hence will be briefly describedhereinafter.

In operation 811, when an object is in contact with the image sensor,the image sensor may acquire a contact image of the object, and inoperation 812, the processor may extract a characteristic point from theacquired contact image of the object. In this case, the characteristicpoint is a reference point within a desired measurement area of theobject and may include, for example, a center point of a fingerprint ora position of a predetermined blood vessel within the measurement area.

When the characteristic point is not extracted (i.e., does not exist) inoperation 812 (operation 813—No), in operation 814, it is determinedwhether the number of times that the position of the image sensor isadjusted for extracting a characteristic point is less than a thresholdvalue.

When the number of times that the position of the image sensor isadjusted is less than the threshold value, in operation 815, theposition of the image sensor may be arbitrarily adjusted by driving theactuator. When the number of times that the position of the image sensoris adjusted is greater than or equal to the threshold value, inoperation 816, the user may be guided for recontact.

When the characteristic point is extracted (i.e., does exist) inoperation 812 (operation 813—Yes), in operation 817, the contactposition and direction of the object may be determined based on theextracted characteristic point. In this case, the contact position anddirection may be determined based on the extracted characteristic pointand the center point of the FOV of the image sensor.

In operation 818, based on the determined contact position anddirection, it may be determined whether the position of the image sensoris to be adjusted.

When the position of the image sensor is to be adjusted, in operation819, the position of the image sensor may be adjusted by driving theactuator.

When the position of the image sensor is not to be adjusted, inoperation 820, a pulse wave signal may be extracted based on the contactimage, and in operation 821, bio-information may be estimated based onthe extracted pulse wave signal.

The current embodiments can be implemented as computer readable codes ina computer readable record medium. Codes and code segments constitutingthe computer program can be easily inferred by a skilled computerprogrammer in the art. The computer readable record medium includes alltypes of record media in which computer readable data are stored.Examples of the computer readable record medium include a ROM, a RAM, aCD-ROM, a magnetic tape, a floppy disk, and an optical data storage.Further, the record medium may be implemented in the form of a carrierwave such as Internet transmission. In addition, the computer readablerecord medium may be distributed to computer systems over a network, inwhich computer readable codes may be stored and executed in adistributed manner.

A number of examples have been described above. Nevertheless, it will beunderstood that various modifications may be made. For example, suitableresults may be achieved if the described techniques are performed in adifferent order and/or if components in a described system,architecture, device, or circuit are combined in a different mannerand/or replaced or supplemented by other components or theirequivalents. Accordingly, other implementations are within the scope ofthe following claims.

What is claimed is:
 1. An apparatus for estimating bio-information, theapparatus comprising: an image sensor configured to acquire a firstcontact image of an object, based on the object being in contact withthe image sensor; an actuator; and a processor configured to: determinea contact position and a direction of the object, based on the acquiredfirst contact image; and control the actuator to adjust a position ofthe image sensor, based on the determined contact position and thedetermined direction of the object so that a field of view (FOV) of theimage sensor moves to a predefined measurement area on the object. 2.The apparatus of claim 1, wherein the processor is further configuredto: extract a characteristic point of the object, from the acquiredfirst contact image; and determine the contact position and thedirection, based on the extracted characteristic point.
 3. The apparatusof claim 2, wherein the characteristic point of the object comprises afingerprint center point of a finger.
 4. The apparatus of claim 2,wherein the processor is further configured to: determine whether theextracted characteristic point exists in the first contact image; andbased on the extracted characteristic point being determined to notexist in the first contact image, guide a user to bring the object intocontact with the image sensor.
 5. The apparatus of claim 2, wherein theprocessor is further configured to: determine whether the extractedcharacteristic point exists in the first contact image; and based on theextracted characteristic point being determined to not exist in thefirst contact image, repeat a predetermined number of times ofcontrolling the actuator to adjust the position of the image sensor toan arbitrary position and then extracting the characteristic point, fromthe acquired first contact image acquired after the position of theimage sensor is adjusted.
 6. The apparatus of claim 2, wherein theprocessor is further configured to: determine whether the extractedcharacteristic point exists in the first contact image; and based on theextracted characteristic point being determined to not exist in thefirst contact image, estimate a position of the characteristic point bycomparing a reference contact image and the acquired first contactimage.
 7. The apparatus of claim 1, wherein the processor is furtherconfigured to: determine a displacement of the image sensor, based onthe determined contact position and the determined direction; andcontrol the actuator to adjust the position of the image sensor, basedon the determined displacement.
 8. The apparatus of claim 1, wherein theimage sensor is further configured to acquire a second contact image ofthe object, based on the position of the image sensor being adjusted,and the processor is further configured to: extract a pulse wave signal,based on a pixel intensity of the acquired second contact image; andestimate the bio-information, based on the extracted pulse wave signal.9. The apparatus of claim 8, wherein the processor is further configuredto: generate an oscillogram, based on the extracted pulse wave signaland a contact pressure of the object; and estimate the bio-information,based on the generated oscillogram.
 10. The apparatus of claim 9,further comprising a force/pressure sensor configured to measure acontact force or the contact pressure that is applied between the objectand the image sensor, based on the object in contact with the imagesensor changing a force.
 11. The apparatus of claim 9, wherein theprocessor is further configured to acquire the contact pressure, basedon the pixel intensity of the acquired second contact image and using apredefined contact pressure conversion equation.
 12. The apparatus ofclaim 8, wherein the bio-information comprises any one or anycombination of a blood pressure, a vascular age, an arterial stiffness,an aortic pressure waveform, a blood vessel elasticity, a stress index,and a degree of fatigue.
 13. A method of estimating bio-information, themethod comprising: acquiring, by an image sensor, a first contact imageof an object, based on the object being in contact with the imagesensor; determining, by a processor, a contact position and a directionof the object, based on the acquired first contact image; andcontrolling, by the processor, an actuator to adjust a position of theimage sensor, based on the determined contact position and thedetermined direction of the object so that a field of view (FOV) of theimage sensor moves to a predefined measurement area on the object. 14.The method of claim 13, wherein the determining of the contact positionand the direction comprises: extracting a characteristic point of theobject, from the acquired first contact image; and determining thecontact position and the direction, based on the extractedcharacteristic point.
 15. The method of claim 14, further comprising:determining whether the extracted characteristic point exists in thefirst contact image; and based on the extracted characteristic pointbeing determined to not exist in the first contact image, guiding a userto bring the object into contact with the image sensor.
 16. The methodof claim 14, further comprising: determining whether the extractedcharacteristic point exists in the first contact image; and based on theextracted characteristic point being determined to not exist in thefirst contact image, repeating a predetermined number of times ofcontrolling the actuator to adjust the position of the image sensor toan arbitrary position and then extracting the characteristic point, fromthe acquired first contact image acquired after the position of theimage sensor is adjusted.
 17. The method of claim 14, furthercomprising: determining whether the extracted characteristic pointexists in the first contact image; and based on the extractedcharacteristic point being determined to not exist in the first contactimage, estimating a position of the characteristic point by comparing areference contact image and the acquired first contact image.
 18. Themethod of claim 13, wherein the controlling of the actuator comprises:determining a displacement of the image sensor, based on the determinedcontact position and the determined direction; and controlling theactuator to adjust the position of the image sensor, based on thedetermined displacement.
 19. The method of claim 13, further comprising:acquiring a second contact image of the object, based on the position ofthe image sensor being adjusted; extracting a pulse wave signal, basedon a pixel intensity of the acquired second contact image; andestimating bio-information, based on the extracted pulse wave signal.20. The method of claim 19, wherein the estimating of thebio-information comprises: generating an oscillogram, based on theextracted pulse wave signal and a contact pressure of the object; andestimating the bio-information, based on the generated oscillogram.